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ABSTRACT 



A method for dynamically sharing an application in a 
conference system while maintaining state regardless of 
whether the application is under execution in all conference 
devices. The conference system includes a plurality of 
conference devices with at least one conference device 
executing an application program. The method for dynami- 
cally sharing the application program among the plurality of 
conference devices in the conference system while main- 
taining state, includes the steps of: communicating a request 
to share the application program from a requesting confer- 
ence device which is executing the application to all enrolled 
conference devices; storing an executed state of the appli- 
cation program to be shared in the requesting conference 
device; starting the application program to be shared by 
enrolled conference devices in which an application pro- 
gram corresponding to the application program to be shared 
is not started; communicating the executed state stored in the 
requesting conference device to all enrolled conference 
devices; and processing by all enrolled conference devices 
of the application program to be shared in a shared executed 
state equal to the executed state communicated. 

5 Claims, 14 Drawing Sheets 
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SYSTEM AND METHOD FOR 
DYNAMICALLY SHARING AN 
APPLICATION PROGRAM AMONG A 
PLURALITY OF CONFERENCE DEVICES 
WHILE MAINTAINING STATE 

RELATED APPLICATIONS 

This application is a continuation of U.S. patent applica- 
tion Ser. No. 08/371,915, filed Jan. 12, 1995, now aban- 
doned. 

FIELD OF THE INVENTION 

The present invention relates to a conference system 
control method, a conference device, and a conference 
system, and, more particularly, to a conference system 
control method, a conference device, and a conference 
system for opening a conference by using a plurality of 
conference devices connected by a network or communica- 
tion line such as a LAN, ISDN, or telephone line. 

BACKGROUND OF THE INVENTION 

In a conference system, each of a plurality of users 
connects each terminal to a line or network to open a 
conference while transferring data between terminals. In this 
type of system, conference systems, for realizing an appli- 
cation program (hereafter referred to as an application) 
which can be shared by all users are roughly divided into 
"centralized" and "decentralized." 

The centralized system allows data to be input to an 
application from a plurality of user terminals and transmits 
output data to all user terminal. Rapport uses a centralized 
system (S. R. Ahuja, J. R. Ensor, and D. N. Horn, "The 
Rapport Multimedia Conferencing System," Proc. COIS 88, 
pp. 1-8, 1988). 

However, the centralized system is not preferable because 
much data is output from the application, increasing the load 
on hardware. Moreover, the centralized system is restricted 
in that the system must be applied to a server-client window 
system in which display processing and control are inde- 
pendently performed like the X-WINDOW SYSTEM which 
operates on UNIX. 

The decentralized system ensures the same state by 
executing the same application in each user terminal and 
cooperatively operating each application. The system is used 
for ASSOCIA (Yoshiyuki Nakayama et al., "A Computer- 
Supported Multiparticipant Realtime Teledialogue System/* 
Thesis, Journal of the Information Processing Society of 
Japan, Vol. 32, No. 9, pp. 1190-1199, 1991) and MMConf 
(T. Crowley et al., "MMConf: An Infrastructure for Building 
Shared Multimedia Applications," Proc. CSCW 90, pp. 
329-342, 1990). 

ASSOCIA keeps a plurality of instances (entities) of an 
application in the same state by ensuring at the system side 
that all data values input to all applications from a plurality 
of users coincide with each other in a time series. Therefore, 
it is possible to share the "instances" of all applications and 
keep them in the same state when start and execution are 
simultaneously performed in a plurality of user terminal. 

With ASSOCIA, however, when a user by whom an 
application currently executed is not shared is going to share 
the application, the state of an application of a user who 
already shares the application may not coincide with the 
state of an application of a user who is going to share the 
application in the middle. That is, the application currently 
executed has already-read execution riles and states such as 
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attribute values of the present application (e.g. cursor 
location, expansion or contraction, and fonts). Therefore, 
even if the same applications are started by other user 
terminals at the initial state and the same input is given to 

5 applications after they are shared, it is impossible to realize 
a shared application which coincides with any other appli- 
cation because each state when the application is shared 
differs. Therefore, to keep a currently-executed application 
in a consistent state when the application is shared, a 

10 low-level consistency of giving the same input is insuffi- 
cient. 

Also, in the case of MMConf, it is possible to keep a 
plurality of manifestations of applications simultaneously 
started in the same state but it is impossible to keep a shared 

15 application consistent because processing for sharing an 
application is not assumed. 

As another example of the above-mentioned, the official 
gazette of Japanese Patent Laid-Open No. Hei 3-192845 
discloses a method for controlling an interaction system 
serving as a conference system, which realizes a dialogue by 
the fact that a plurality of users accesses a network of 
terminals. This method realizes a dialogue equivalent to a 
conference by processing participation in the dialogue and 
the exit of a user who can attend a conference (enrollee) 

25 from the dialogue by each terminal so that participation and 
exiting can be done easily. 

However, the above interaction system control method 
only processes the participation in and exit from a dialogue 

30 equivalent to a conference. Therefore, it is impossible to 
keep a shared application consistent at the start of a confer- 
ence and during the conference because processing for 
sharing an application is not assumed. 

Moreover, the official gazette of Japanese Patent Laid- 

35 Open No. Hei 4-186456 discloses a common information 
processing system serving as a conference system making it 
possible for a plurality of users to perform a dialog and 
common information processing by connecting terminal to a 
network or line. This system allows each user to process 

40 electronic information in common with other terminals by 
performing a dialogue while referencing the same output 
result and so on. 

However, the common information processing system 
also has a restriction in that the system must be applied to a 
45 server-client window system in which display processing 
and control are independently performed like the 
X-WINDOW SYSTEM which operates on UNIX. 

Thus, there is a need for a more cost-effective conference 
system and method which allows dynamic sharing of appli- 
5 P cations regardless of whether a user is currently executing 
the application. The present invention addresses such a need. 

SUMMARY OF THE INVENTION 

5S It is therefore an object of the present invention to provide 
a conference system whereby all user terminals executing a 
common application are maintained in a same state. 

It is yet another object of the present invention to bring 
user terminals joining a conference already in progress to a 

60 same state as other user terminals in the conference. 

According to the invention, a plurality of individual user 
terminals are joined together by a network. In operation, a 
requesting user terminal runing an application program 
requests a conference. While the application program may 

65 be of any type, for the purposes of illustration, a blackboard 
program is run whereby conferees attending the conference 
can write on the blackboard and erase the blackboard just as 
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a conferee might do when physically attending a conference 16, to 16 K . . . Terminal units 

in a conventional conference room. The requesting user 18 3 to 1S X . . . Users 
terminal issues a request to share the application program 

with all conferee user terminals causing all conferee user DETAILED DESCRIPTION OF THE 

terminals to load the application program. The requesting 5 PREFERRED EMBODIMENT 

user terminal .ben transmit ihe current state of the appli- EmbodimeD u of the present invention are described 

cation program to the conferee user termtnals such that all be]ow „ refcrenci lhe F accompanying drawings . 

user terminals are brought to the same state and the confer- A . • « * A . . 

ence can proceed with all conferees seeing the same things n M sbow ?. m FIG X - a conference system 10 which is the 

on their individual terminals. 30 firsl embodime nl comprises an annular network 12 for 

a Hi- 1 .1 * 1 j _. r sending/receiving a digital signal and K (a natural number) 

A callback protocol memory b also provided for stoncg u lQ \ 6 |^*d ^ > 

a processing program and identifiers corresponding to even c * 1 n , c ^ * - , • j . 

. K u u u * * r • conference system 10, transfer data is transmitted through 

information whereby when a state of the application is t . ^ , . *? u ♦ • 1 1* * 1* * . j iT 

, , . , r 1 c ■ the network 12. Each terminal 16, to 16^ is operated by 

changed, such as by, for example, a conferee writing a 10 . n 10 tu * 1 11 • . . ■ . j . .1/ 

*u ui i k 1 t t . . fi . ,. 4 . nc uscrs 18 i t0 Th e network 12 is not restricted to the 

message on the blackboard, the new state of the application is 1 * . 

. , & , . „ V . . above networks (Ethernet and Token Ring Network). It is 

is transmitted to all user terminals participa ing in the „ n , P -u fl lrt 11PO .,- , , . , , , ; , 

P , t „ , , . *j ... possible to use a public telephone network, a network such 

conference such that all such terminals maintain a same V. jcfmsj « u* *• *u lu » ^ 

slate as ISDN, or a combination of them as the network 12. 

Moreover, each terminal 16 1 to 16^, stores a conference 

BRIEF DESCRIPTION OF THE DRAWINGS 20 room system 11 as an execution program for progressing a 

™~ - . • • . 1 1 1 • , . c conference through terminals among a plurality of users as 

FIG. 1 is a schematic block diagram showing the confer- described later 

ence system of the first embodiment; 4 ." . 

™ 0 . . • • ii 1 j • u -i ^ shown in FIG. 2, the terminal 16, (i:lii^K) com- 

FIG. 2 is a schematic block diagram showing a terminal „ , ' /u ft c , . ™^v^ 

f c . . 4 pnses a personal computer (hereafter referred to as a PC) 22, 

of the first embodiment; L ■ . « j* 1 n • j- 1 J 

' 25 serving as a controller, a display 24,- serving as a display, and 

FIG. 3 is a block diagram showing the rough internal keybo ard 26, serving as an input unit. In addition to the 

constitution of the computer of a terminal of the first above constitution, it is possible to connect a liquid crystal 

embodiment, display or projector serving as a display, a mouse serving as 

FIG. 4 is a corresponding diagram showing types of a coordinate input unit, a camera serving as an image input 

conference events, names, and parameters; 3Q unit, a microphone serving as a voice input unit, and a pair 

FIG. 5 is a flowchart showing the flow of a start for of speakers serving as voice output units. The microphone 

dynamically incorporating a common application; and the two speakers can be replaced with a head set such 

FIG. 6 is a flowchart showing the flow of processing in a as a telephone set or transceiver. Moreover, it is possible to 

requesting terminal when a plurality of users shares a xi tne lwo speakers so as to produce a stereophonic effect 

common application; 35 or it is possible to use three or more speakers for multichan- 

FIG. 7 is a conceptual view showing the flow of process- nel reproduction or use one speaker for monaural reproduc- 
ing in a requesting terminal when a plurality of users shares tl0n - In P ut Processing can be made on a CRT without using 
a common application- a keyboard by connecting a mouse and voices can be input 

FIG. 8 is a flowchart showing the flow of processing in a 0T AA ° Utpul by u ™ ctiQ g a microphone and a speaker. In 

requested terminal when a plurality of users shares a com- 40 ad ^ 10Q 10 the above constitution, it is possible to connect 

mon application; a dl 6> uzcr serving as a coordinate input unit, a printer 

„ . ' , . , . , „ r serving as an output unit, and a scanner serving as an input 

rlG. 9 is a conceptual view showing the flow of process- umt 

ing in a requested terminal when a plurality of users shares * Dr> „ . _ nT . - D . _ A . _ „ 

* ™ rtri „ i- fltl - . J A PC 22, comprises a CPU 28., ROM 30,, RAM 32,, 

a common application; _ ' , r . . 

■ L L i 1 ,* . . 45 memory 34„ and an input/output port I/O 36, connected by 

FIG. 10 is a schematic block diagram showing the con- a bus 38 ^ ^ daU afld commands can ^ lraDsferre i 

terence system ot the second embodiment; between them. Memory 34, can use external memory such 

FIG. 11 is a conceptual view showing the flow of pro- as a floppv disk drive> hard disk drivC( or optoelectronic 

cessing in an integrated controller when a plurality of users memory. The input/output port I/O 36, is connected to the 

shares a common application in the second embodiment; 50 ne twork 12 through a communication interface I/F 37,. and 

FIG. 12 is a conceptual view showing the flow of pro- also connected to the display 24, and keyboard 26.. 

cessing in a request-side terminal unit when a plurality of , n FIG 3^ tbc outline of lbe internal constitution of the PC 

users shares a common application in the second embodi- 22 . for realizing the conference room system 11 in the 

menl * terminal 1 6- is shown in the form of a block diagram for each 

FIG. 13 is a conceptual view showing the flow of pro- 55 function. The PC 22,- has a communication control section 
cessing in a requested terminal when a plurality of users 
shares a common application in the second embodiment; 

FIG. 14 is a schematic block diagram showing a confer- later) for controlling conference operations such as opening 

ence system for separately sending or receiving voice and and closing of a conference (hereafter referred to as a 

animation information; and eo conference room) opened among a plurality of terminal and 

FIG. 15 is a schematic block diagram showing a different an enrollee's entering and exit the room, a common appli- 

conneclion configuration among a plurality of terminals in a cation section 50 ( - comprising one or more programs which 

conference system. are shared by enrol lees in the conference room and concur- 

DcscriptionofSvmbols rently used (e.g., a program or the like of a common 

65 blackboard on which a character or a diagram can be written 

10 . . . Conference system or on which a character or diagram can be erased), and a 

12 . . . Network common application management section 44,.. The common 



40\ for communicating data or the like with another PC ; 
(jrl^j^K, i*j), a conference control section 42 f (described 
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application management section 44, includes a callback 
routine section 48,- comprising a processing program 
(hereafter referred to as a callback routine) to the common 
application section 50, for various instance processing 
(hereafter referred to as conference events; described later) s 
in the case of the call of a common application service 
section 46 ( , and the common application section 50,. or 
operations of a conference. The common application service 
section 46, stores a processing routine included in a common 
application (e.g., a program for storing a state of a common 
application), whicb is called by an individual common 
application or by the common application service section 46 ; 
(i*j) of another terminal. 

The communication control section 40,- is connected to the 
network 12 in order to exchange data with a plurality of 35 
terminal used for one conference through a wide-area net- 
work such as a LAN or ISDN and also connected to the 
conference control section 42,- and the common application 
section 50,-. The communication control section may per- 
form one-to-one data transfer to and from another terminal 2Q 
separately from an opened conference. The conference con- 
trol section 42,- is connected to the common application 
section 50,-, common application management section 44„ 
and keyboard 26,. The common application section 50,- is 
connected to the common application management section 
44, 

The conference control section 42, is a control section for 
processing a request for generation or deletion of a confer- 
ence room, communicating with the conference control 
section 42 ; - of another terminal to process the acceptance of 30 
the request, and performing processing so as to synchro- 
nously keep the same contents relating to new enrollees in 
a conference and to the fact that a user having an application 
operating sends a conference event such as replacement or 
the like to the common application section 50,-. For example, 35 
the section 42; manages a conference member as a enrollee 
to a conference and the common-application use right and 
has a function for notifying the common application of a 
change in the number of conference enrollees, of moving the 
use right to a conference enrollee as a conference event. 40 

Thus, for the conference system 11 of this embodiment, 
the conference control section 42,. for operating a conference 
and the common application section 50,- which is a confer- 
ence event section simultaneously used by enrollees in a 
conference and stores common applications are indepen- 45 
dently constituted. 

As shown in FIG. 4, the conference event CE shows 
details of conference state change. The conference event CE 
comprises an event name and a parameter. Corresponding to 
the conference event CE, a callback routine for processing 50 
the conference event CE concerned is previously registered 
in the callback routine section 48,-. In FIG. 4, event names 
TERMINATE, NEW, OPEN, CLOSE, and DELETE are 
shown as the conference event CE for basic conference 
operations such as termination of conference processing, 55 
new conference, opening of conference, closing of 
conference, and deletion of conference; event names 
BEGINSHARE and ENDSHARE are shown as the confer- 
ence event CE for sharing a requested common application 
such as beginning of assignment and ending of assignment 60 
for sharing; event names ENROLLED and UNENROLLED 
are shown as the conference event CE for changing enrollees 
in a conference room such as addition and deletion of 
enrollees; event names ENTER-START, ENTER- 
COMPLETE, LEAVE-START, and LEAVE-COMPLETE 65 
are shown as the conference event CE for newly participat- 
ing and exiting a conference room such as new participation 
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and exiting; and event names FLOOR-REGISTERED, 
FLOOR-UNREGISTERED, FLOOR-ENQUEUED, 
FLOOR-RELEASED, FLOOR-OBTAINED, AND 
FLOOR-CANCELED are shown as the conference event 
relating to the conference room control right such as entry, 
deletion, assignment, release, acquisition, and cancellation 
of the control right. 

Parameters serving as arguments of event names as the 
conference events include toolNumber, toolID, confID, 
userlD, and floorlD. The parameter toolNumer corresponds 
to an identification number corresponding to a type of 
common application, the parameter toolID corresponds to an 
identification number when using a plurality of common 
applications for one conference under different states, the 
parameter confID shows a conference room number (e.g., 
serial number), the parameter userlD corresponds to a user's 
identification number, and the parameter floorlD corre- 
sponds to an identification number corresponding to a type 
of control right. 

As shown in FIG. 3, the conference control section 42, 
comprises a conference control request processing section 
52,-, a conference event processing section 54„ a common 
application registration section 56/, a conference event dis- 
tribution section 58,, a conference event reception section 
60,., a common application table 62/, a callback routine table 
64,-, and a conference table 66,-. 

The conference control request processing section 52,. 
processes a request for conference state change from the 
terminal of its own or another terminal. The conference 
event processing section 54,- generates or delivers a confer- 
ence event corresponding to a conference state change. The 
common application registration section 56,- dynamically 
registers a common application at start or the like. The 
conference event distribution section 58,- distributes and 
outputs a conference event for a conference state change 
request to enrollees who can attend the conference. The 
conference event reception section 60,- receives a conference 
event for a conference state change request from another 
terminal. 

The common application table 62,. is provided with serial 
numbers for common applications and stores serial numbers, 
common application names, the address of the module 
concerned where one common application is loaded in RAM 
(memory) as a module, and the address of the common 
application initialization routine. The callback routine table 
64,- stores common application serial numbers in the com- 
mon application table 62,., event names, and call back 
routine address. The conference table 66,- stores the number 
for a conference room which is currently opened and in 
which the user 18,- can participate, the conference room 
name, and the list of users who can use the conference room. 
Memory in which common applications are loaded is not 
restricted to RAM. It is possible to separately connect 
memory so that data is transferred to and from a terminal or 
add EMS memory or a cache memory. In this case, it is also 
possible to form main memory by adding EMS memory or 
cache memory to the terminal so that data can be loaded. 
This constitution is preferable because the device constitu- 
tion is simplified. 

A common application name (e.g., name of a dynamic 
link library to be loaded) used by a user in the conference 
room system 11 is stored beforehand in the common appli- 
cation management section 44, as an application file APR 

The following is a description of the functions of this 
embodiment. For each terminal described below, a confer- 
ence room to be opened (used) when starting a conference 
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program references the conference table 66,. That is, a user The above processing is repeated for all common appli- 

who can attend a plurality of conference rooms is repeatedly cation names stored in the application file APR 

entered in the conference table 66,. Therefore, for example, Therefore, the conference control section 42, defines 

one or more conference rooms are selected by displaying the con f erence state change details as conference events (FIG. 

conference room list in the conference table 66, or a plurality 5 4) ^ ^ defined conference events t0 the common 

of conference rooms is selected by default at the start of the appUcatio n xtiion 44 mieby it * possibIe f or a plurality 

program. One field is registered in the conference table 66, of confercncc evcnts of a common application to keep a 

when a conference device is held. For example, when the consistcnt state independent of the conference state change, 

user 18, opens a conference room an identification number ^ correspond to window events in a window system, 

is given to the conference room and a conference room name 30 Each common appIication registers ils own callback routine 

corresponding to the identification number and a list of users in , hc controi 42 for a con f ere oce event 

who can attend the conference room are stored. It is pref- M b each common a lication t0 keep conf e rcn ce 

erable lo obtain consent from persons who can attend the event consislenc for each terminaI m the initialization 

conference room before storing a list of them. Though a romine of each commoQ lication When confer ence 

plurality of conference rooms can be registered in the 35 slales cfa the conference control ^tioa 42, calls the 

conference table 66,, the use of a single conference room use caUback f0Utine re for the inference event, 

is described below to simplify the description. Executing a j^tty it is possible to dynamically incorporate common 

plurality of conference rooms on a terminal is to process a applications , which are for a conference necessary at the 

plurabty of conference rooms by means of individual time- sM of a conference program without changing the states of 

shanng one by one. Therefore, this processing corresponds 20 , hc conference 

to the processing of each conference room by means of , , , . 

• j- m 1 u • • # ~ . ,, j • j A method for shanng a common application currently 

individual time-sharing interrupts or so-called window pro- , . . • , * * 

„ • n i-^i,- n j„ n(r _„c_ „ „ executed in the terminal of a user by all conference enrollees 

cessine displaying a plurality or conterence rooms on a . , , _ J - 

j 1 »• 1 • a f m in the same state independent or a conference room currently 

screen and selectively using a specific conference room. ... ..... . r r^,™ ^ « ^ 

Ljf t. - i.L t* opened 1 s described below by referencing FIGS. 6 to 9. To 

First, a method for obtaining only the common apphca- 25 • re .l j ■ *• ■ j .u 7 j * 

' » . . r i simplify the description, it is assumed that a common editor 

lion necessary for operating the conference concerned or a „ r . . ' . r c 

,. . , . , . e S which is the common editor S serving as a common 

common apphcat.on to be executed only by each user from licaljon loca „ execuled onl at lhe terminal 16 of a 

a plurality of common applications mcluded m the common £iA\n* conference room opened by three users A, 8, and 

application section 50, and how the conference control n . . , . t . t . a xlc . ar J lt • J cn nnpci ,. tn » cc „ ma 

rr . rL . tj... jr Cis snared by the three users, lt is also possible to assume 

^at^v^tm^^ta^^o«^ 30 tfaat com J OD licalion executed £ nly by one ^ 

application are described below together with Starting room k shared b three users 

routine in FIG. 5. . , , ; ' e , 

. / ♦ u \r no r According to input through a keyboard 26. of the user A, 

When a power switch (not shown) of a PC 16, of a t _ & . f . 6 - J . \ . 

, j \u . • , ,/ • j C a request for sharing the conference event of the common 

terminal is turned on and the terminal 16, is ready for M „ . . .. f , , A <* 

. , c . ii r c • editor S is output to a conference control section 42. of the 

operation and the conference room system 11 for per forming 35 , . , v c . A/t - n - v ™. * . 

r . ■ c c m j * • . ■ terminal 16. of the user A (step 202). This request is input 

operations such as opening of a conference and participation . f A . i . * 

. \ . - . r 4 . j.. • * . ■ . . r to a conference control request processing section 52. 

in the conference is started through a keyboard, the common r s i nal ath Gl FIG T> 

application registration section 56* of the conference control V si g na pa » • 

section 66, reads one common application name from the ^ conference control request processing section S2 A of 
application file APF storing common application names 40 the user A generates a conference event CE with the event 
(step 102). Then, it is determined whether all common name BEG INSH ARE necessary for processing and outputs 
application names stored in the application file APF are read the conference event (event name: BEGINSHARE) to a 
by determining the end of the application file APF (step conference event distribution section 58^ (step 204, signal 
104). In the case of a negative determination, a module P ath G2 )- ^ conference event distribution section 5S A 
stored in memory 34* corresponding to a common applica- 45 references a conference table 66 A (signal path G3 in FIG. 7) 
lion with the read name is loaded in RAM (memory) by the t0 determine that users A, B, and C are enrollees at a 
common application registeration section 56, (step 106). The conference room and Attributes the conference event to 
common application' registration section 56, gives a serial these three users (step 206). The conference event is also 
number to the common application with the read name and distributed to the user A who requested sharing of the 
enters the serial number, the common application name, the 50 conference event and the transfer data designating users B 
address of the module loaded in RAM (memory), and the and C 15 out P ut t0 ! ermina! 16 * and 16 c of users B and C 
address of an initialization routine in the common applica- lhrou g h a communication control section 40 A and the net- 
lion table 62, (step 108). The initialization routine is located work 12 ( S1 &^ P alh G4 )- 

at a predetermined location (e.g., first opening function) of Conference event reception sections 60^, 60^, and 60 c of 

the loaded module, which sets a default and includes the 55 users A, B, and C receive a conference event. Details of 

address of a callback routine corresponding to a conference processing for users B and C are described later. The 

event included in a common application. Then, the common conference event received by the conference event reception 

application registration section 56, calls the initialization section 60^ is output to a conference event processing 

routine of the common application with the read name (step section 54^ (step 208, signal path G5). 

110) and obtains the address of a callback routine corre- 60 The conference event processing section 54^ reads a 

sponding to a conference event from the called initialization conference number and common application number from 

routine (step 112). Then, the common application registra- the parameter of the input conference event(depicted in FIG. 

tion section 56 ( establishes a correspondence between the 4), references a callback routine table 64 A (signal path G6) 

address of the callback routine thus obtained, a common to obtain the address of a callback routine corresponding to 

application number and an event name and enters the 65 the conference event of the common application, and 

common application number, event name, and callback requests starting of the callback routine (signal palh G7). A 

routine address in the callback routine table 64, (step 114). common application management section 44 A calls the 
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requested callback routine and starts it (step 210). The Synthetically, a request for sharing the conference event 

conference event processing section 54^ waits for the next of the common editor is issued to the conference control 

processing until the processing of the callback routine section of the user A from the user A, the conference control 

terminates. This processing is executed in the callback section of the user A communicates with conference control 

routine of the common editor independently of the confer- 5 sections of users B and C, and a callback routing for sharing 

ence control section. lne conference event of the common editor S is called by 

The callback routine of the common application orders terminals of users A, B, and C The callback routine of the 

the common editor S of the user A to terminate processing common editor of the user A communicates with the caU- 

(signal path G8) and the common editor S terminates ^^^'f^u™^ f^l ° f * f \ nd C ™ C 

processing currently being executed (step 212). 10 common cdi ;° r lt of the tennm ^ <? f lhe user A first terminates 

„„ .. . . - . ' execution of all processing before sharing to wait for the 

vynen receiving termination ot processing, the common sUle tQ become stead and thereafter 

starts the common 

application section of the user A stores the current state of editor with terminals of ^ fi and c M[CJ {{ fa confirmed 

the common editor S (e.g read data file portion currently tnat the common editor fe slarted b Qf ^ B and 

edited or cursor location) and outputs the stored state to Q the stale of lhe common edhor g fa ^ tQ users 

users B and C (step 214, signal path G9). « fi c When ^ ^ pfocessing terminateSj users ^ ^ 

After state output to users B and C terminates, the and C terminate callback routine processing. When caUback 

common application section 50* outputs a termination sig- routine processing terminates and control returns to the 

nal to the conference event processing section S4 A to ter- conference control section, common editors S in the same 

mmate processing of conference event which is conference state are working in terminals of three users. Thereafter, it is 

control in the conference event processing section 54 A (step possible to input data to the common editor. 

216, signal path G10). [{ ^ t0 share a common application under 

Processing by terminals 16 B and 16 c of users B and C is execution by clearly separating conference control section 

described below. Because almost the same processing is processing from that of the common application, managing 

performed for users B and C, processing by the terminal 16 B 2S the conference event notification of sharing a common 

of the user B is described below for simplicity. application by the conference control section, providing the 

When a signal based on transfer data is input to the common application with functions for storing the current 

conference event reception section 60^ from the terminal of execution state and realizing a new state, and shifting to the 

another user (signal path G20, FIG. 9), the sharing routine same state by synchronizing common terminal applications, 

in FIG. 8 is executed and the conference event reception 30 The second embodiment is described below. In the first 

section 60^ receives a conference event. The conference embodiment, a case is described in which only the terminal 

event received by the conference event reception section 60 fl of each user participating in a conference is connected by a 

is output to the conference event processing section 54 B network or the like. However, the present invention is not 

(step 222, signal path G21). restricted to the above case. As shown in FIG. 10, it is also 

The conference event processing section 54 B reads a 35 possible to manage a conference by connecting one or more 
conference number and a common application number from centralized managing devices 17 to a network or the like, 
the parameter of the input conference event and references For the second embodiment, the same portion with that of 
the conference table 66 g (signal path G22a) and the callback the first embodiment is provided with the same symbol and 
routine table 64 B (signal path G22) to obtain the address of its detailed description is omitted. Therefore, only portions 
a callback routine corresponding to the conference event of 40 different from those of the first embodiment are described 
a conference room in which a common application is below. The conference system 10 of this embodiment, as 
included and requests starting the callback routine (signal shown in FIG. 10, comprises an integrated controller 17 
path G23). A common application managing section 44 B comprising a host computer such as a microcomputer con- 
calls the requested callback routine to start it (step 224). The nected to a network 12 and L (L is a natural number) 
conference event processing section 54 B waits for the next 45 terminal 16 1 to 16 L . 

processing until processing of the callback routine termi- As shown in FIG. 11, the integrated controller 17 corn- 
nates- prises a communication control section 41 for communicat- 

The callback routine of a common application designates ing data with a terminal and a conference control section 43 

starting the common editor S serving as a common appli- for controlling conference operations such as activation and 

cation corresponding to the common editor S of the user A 50 termination of a conference room opened among a plurality 

(signal path G24) and starts the common editor S at the of terminals and enrollees entering and exiting the confer- 

lerminal B of the user B (step 226). In this case, it is ence. The communication control section 41 is connected to 

preferable for the common application to store all current the network 12 and also to the conference control section 43. 

states or accept a state sent from another unit as a new state The conference control section 43 comprises a conference 

to realize sharing during execution. 55 control request processing section 53, a conference event 

When receiving termination of a start, the common appli- distribution section 59, and a data storage section 51 for 
cation section of the user B waits until reception of transfer holding an executed state of a common application. The 
data to be shared sent from another user terminates (step conference control request processing section 52, generates 
228, signal path G25). When reception of transfer data a conference event corresponding to a change in the con- 
terminates, the common editor of the user B performs 60 ference state when a request for changing conference states 
processing so as to coincide with the stale of the common is output from various terminals. The conference event 
editor S of the user A (step 230). After processing terminates, distribution section 58,- distributes and outputs a conference 
the common application section 50^ outputs a termination event for a conference state change request to enrollees in 
signal to the conference event processing section 54^ to the conference. 

terminate processing of the conference event which is con- 65 A terminal X6 A comprises a PC 22 A , a display 24 A and a 

ference control in the conference event processing section keyboard 26^ like that of the first embodiment. Because 

54 n of the user B (step 232, signal path G26). other terminals have the same constitution, their description 
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is omitted. As shown in FIG. 12, the PC 22 A of the terminal 
16^ mainly comprises a communication control section 40^ 
for communicating data with another PC, (j:l^j^L, i *j) 
and the integrated controller 17, a conference control section 
42 A (described later) for executing a conference room 5 
opened at a terminal, a common application section 50 A 
comprising one or more programs shared by users and 
concurrently used in a conference room, and a common 
application management section 44 A . The communication 
control section 40 A is connected to the network 12 and a 
keyboard 26 A and also connected to the conference control 
section 42 A and the common application section 50^. The 
communication control section may perform one-to-one 
data transfer to and from other terminals separately from an 
opened conference. The conference control section 42 A is 
connected to the common application section 50^ and com- 
mon application management section 44 A . The common 
application section 50 A is connected to the common appli- 
cation management section 44 A . 

The conference control section 42 A comprises a common 20 
application registration section 56^ a conference event dis- 
tribution section 58 A , a conference event reception section 
60 A , a common application table 62 A , a callback routine 
table 64 A , and a conference table 66 A . 

The conference event processing, section 54 i delivers a 2 s 
conference event corresponding to a conference slate 
change. The common application registration section 56,. 
dynamically registers a common application at starting or 
the like. The conference event reception section 60,- receives 
a conference event for a conference state change request sent 30 
from the integrated controller. 

The functions of this embodiment are described below. 
For each terminal of this embodiment, a conference room 
opened (used) by starting a conference program is per- 
formed by referencing a conference event and the confer- 35 
ence table 66,. from the integrated controller. That is, a user 
who can use a plurality of conference rooms is repeatedly 
entered in the conference table 66,-. Therefore, one or more 
conference rooms axe selected from the conference room list 
of the conference table 66,. by the integrated controller or one 40 
or more conference rooms are selected by default, for 
example, at the start of the program. A field is registered in 
the conference table 66, and the table 66, transmits the 
registered field to the integrated controller 17 when a 
conference is opened. Because it is possible to register a 45 
plurality of conference rooms in the conference table 66,-, a 
case of registering one conference room is described below 
for simplicity. 

First, the start processing of this embodiment is described 
below. Because the start of this embodiment is almost the 50 
same as that of the first embodiment in FIG. 5, it is described 
by referencing FIG. 5 by giving the same number to the 
processing equivalent to that in FIG. 5. When a power 
switch (not shown) of a terminal PC 16,. is turned on, the 
terminal 16,. is ready for operation and the conference room 55 
system 11 for performing operations such as opening of a 
conference and participation in the conference is started by 
a designation through a keyboard, and data is sent to an 
integrated controller. The integrated controller references a 
conference table 67 and sends a conference event showing 60 
the start to the started terminal. The common application 
registration section 56, of the conference control section 66, 
of the terminal 16,- receiving the conference event showing 
the start reads a common application name from the appli- 
cation file APF storing common application names 65 
(corresponding to step 102, FIG. 5). Then, it is determined 
whether all common application names stored in the appli- 
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cation file APF are read by determining the end of the 
application file APF (step 104). In the case of a negative 
determination, a module stored in memory 34,- correspond- 
ing to a common application with the read name is loaded in 
RAM (memory) by the common application registration 
section 56,- (step 106). The common application registration 
section 56,- gives a serial number to the common application 
with the read name and enters the serial number, the com- 
mon application name, the address of the module loaded in 
RAM (memory), and the address of an initialization routine 
in the common application table 62 t - (step 108). The initial- 
ization routine is located at a predetermined location (e.g., 
first opening function) of the loaded module, which shows 
the type of conference and a default and includes the address 
of a callback routine corresponding to a conference event 
included in a common application. Then, the common 
application registration section 56,. calls the initialization 
routine of the common application with the read name (step 
110) and obtains the address of a callback routine corre- 
sponding to a conference event from the called initialization 
routine (step 112). Then, the common application registra- 
tion section 56, establishes a correspondence between the 
address of the callback routine thus obtained and to a 
common application number and an event name and enters 
the common application number, event name, and callback 
routine address in the callback routine table 64,. (step 114). 

The above processing is repeated for all common appli- 
cation names stored in the application file APF. 

Therefore, the conference control section 42,- defines 
conference state change details as conference events (FIG. 
4) and reports defined conference events to the common 
application section 44,.. Thereby, it is possible for a plurality 
of conference events of a common application to keep a 
consistent state independent of the conference state change. 

A procedure for sharing a common application under 
execution is described below by referencing FIGS. 11 to 13. 
Hereafter, to simplify the description, it is assumed that a 
common editor S which is the common editor S serving as 
a common application locally executed only at the terminal 
16 A of a user A in a conference room opened by three users 
A, B, and C is shared by the three users. It is also possible 
to assume that a common application executed only by one 
user outside the conference room is shared by three users. 

According to input through a keyboard 26 A of the user A, 
a request for sharing the conference event of the common 
editor S is output to the integrated controller 17 (signal path 
G32 in FIG. 11 and the signal path 31 in FIG. 12). 

The conference control request processing section 53 of 
the integrated controller 17 generates an conference event 
CE with the event name BEGINSHARE necessary for 
processing and outputs the conference event (event name: 
BEGINSHARE) to a conference event distribution section 
59 (signal path G33, FIG. 11). The conference event distri- 
bution section 59 references a conference table 67 (signal 
path G34 FIG. 11) to determine that users A, B. and C are 
enrollees at a conference room and distributes the confer- 
ence event to these three users. The conference event is also 
distributed to the user A who requests sharing of the con- 
ference event and terminals 16^ and 16 c of users B and C 
through a communication control section 41 of the inte- 
grated controller 17, network 12, and communication con- 
trol sections 4G\, 40 fl , and 4Q C (signal path G35, FIG. 11; 
signal path 36, FIG. 12; signal path 37, FIG. 13). 

Conference event reception sections 60 A , 60^, and 60 c of 
users A, B, and C receive a conference event. The confer- 
ence event received by the conference event reception 



04/20/2004. EAST Version: 1.4.1 



5,805 ; 

13 

sections is outputted to each conference event processing 
section (signal path 38, FIG. 12; signal path 39, FIG. 13). 
Each conference event processing section 54 reads a con- 
ference number and common application number from the 
parameter of the input conference event, references a con- 5 
ference table and a callback routine table (signal path 40, 
FIG. 12; signal path 41, FIG. 13) and obtains the address of 
a callback routine corresponding to the conference event of 
the common application to request starting the callback 
routine (signal path 42, FIG. 12; signal path 42a, FIG. 13). 
Each common application management section calls and 
starts the requested callback routine. Each conference event 
processing section waits for the next processing until the 
processing of the callback routine terminates. Subsequent 
processing is executed in the callback routine of a common 
editor independent of the conference control section. 15 

The callback routine of the common application of the 
user A requesting sharing orders the common editor S of the 
user A to terminate processing (signal path G43) and termi- 
nates the processing of the common editor S currently being ^ 
executed. The common application section of the user A 
receives the termination of processing, stores the current 
state of the common editor S, and outputs the stored state to 
the integrated controller 17 (signal paths G44 and G44a). 
Thereafter, to terminate the processing of the conference ^ 
event which is the conference control in the conference 
event processing section 54^, the common application sec- 
tion 50 A outputs a termination signal to the conference event 
processing section 54 A (signal path G45). 

Then, processing by terminals 16 B and 16 c of users B and 3Q 
C for which sharing is requested is described below. Because 
almost the same processing is performed for users B and C, 
processing by the terminal 16 B of the user B is described 
below for simplicity. 

Base on the conference event sent from the integrated 35 
controller 17, the conference event processing section 54 g 
reads a conference number and a common application 
number from the parameter of the input conference event, 
references a conference table and the callback routine table 
64 B and obtains the address of a callback routine corre- 40 
sponding to the conference event of the conference room 
including a common application to request starting the 
callback routine (signal path G42a). The common applica- 
tion managing section 44 B calls and starts the requested 
callback routine. The conference event processing section 45 
54^ waits for the next processing until callback routine 
processing terminates. 

The callback routine of a common application orders 
starting the requested common editor S (signal path G46) 
and starts the common editor S at the terminal 16 B of the 50 
user B. In this case, it is preferable to set the common 
application so that it can store al! current states and receive 
a state sent from an external unit as a new stale to realize 
sharing during execution. 

By receiving the start termination, the common applica- 55 
tion section of the user B waits until the reception of transfer 
data from the integrated controller 17 terminates (signal path 
G47). The user B then processes the common editor S so that 
the stale of Ihe common editor B coincides with the state of 
the common editor S of the user A. After processing 60 
terminates, the common application section 50 a outputs a 
termination signal to ihe conference event processing sec- 
lion 54^ to terminate processing of the conference event 
which is conference control in the conference event pro- 
cessing section 54^ of the user B (signal path G48). 65 

Ilierefore, the conference control section of each user 
communicates with the integrated controller, the callback 
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routine for sharing the conference event of the common 
editor S by terminals of users A, B, and C, and it is possible 
to perform processing so that common editors in the same 
state are operated in terminals of the three users. 

Thus, because this embodiment manages generation and 
reporting of a conference event sharing a common applica- 
tion by an integrated controller, it is possible to decrease 
additional processing of each terminal and share a common 
application under execution by using fewer execution pro- 
grams and less memory. 

The conference system 10 of the above embodiment also 
makes it possible to send or receive voice information 
including the voice of a user and music and animation 
information including the face of the user as transfer data. In 
this case, it is also possible to send or receive voice and 
animation information through transmission means 13 such 
as a separate line or separate network as shown in FIG. 14. 
Therefore, by sending or receiving voice information and 
animation information through a separate line or network, it 
is possible to decrease data capacity and manage a confer- 
ence in real time even using conference system having voice 
and animation information as data. 

In the above embodiment, a case of using the annular 
network 12 is described. However, as shown in FIG. 15, it 
is also possible to connect a plurality of terminal to one 
terminal 16^ (X=any one of 1 to N) or connect a plurality of 
terminals to an integrated controller 17. 

As processing for making other user attend an opened 
conference, it is preferable to perform attendance request 
processing for asking whether the user attends the confer- 
ence with a terminal 16y for which attendance is requested. 
In this case, it is preferable to set a terminal 16 ( of the 
attendance request side when requesting another user to 
attend the conference so that the terminal cancels the above 
attendance procedure when transfer data showing that other 
user 18 y refuses attendance is returned or no response returns 
because the other user 18 y - does not operate the terminal 16 ; . 
or is out (after a preset time of a timer lapses). 

Moreover, to proceed with the conference, the common 
application use right may come into contention. In the case 
of the above common blackboard, the common blackboard 
use right contends because a plurality of users simulta- 
neously request the common blackboard use right. In this 
case, it is necessary to make users wait their turn to obtain 
the use right by giving priority to the use right. Data showing 
use right permission and the turn are sent to the conference 
event reception section and the common application section 
of each user. Based on this data, the conference event 
processing section allows a user to use the common black- 
board. It is preferable to set up an application program for 
the common blackboard to enable a user to input data with 
an input unit such as a keyboard or mouse and to send data 
for making input by users other than the permitted user 
impossible to prevent erroneous input. 

As described above, the present invention has advantages 
that only a dynamically necessary application among a 
plurality of applications can be incorporated into a real-time 
conference system for multiple enrollees in which applica- 
tions are executed by the conference device of each enrollee 
at the start of the conference system and these applications 
under execution can be started and shared by the terminal of 
another user in the same state. 

Having thus described our invention, what we claim as 
new and desire to secure by Letters Patent is: 

1. In a conference system including a plurality of confer- 
ence devices with at least one requesting conference device 
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executing an application program to be used in a conference, 
a method for dynamically sharing the application program 
among the plurality of conference devices while maintaining 
a same state of the application program at each of the 
plurality of conference devices, said method comprising the 5 
steps of: 

communicating a request to share the application program 
from said requesting conference device executing the 
application program to other ones of said conference 
devices enrolled in the conference; 10 

storing a current state of the application program in the 
requesting conference device; 

loading the application program by said other ones of said 
conference devices enrolled in the conference; 1S 

communicating the current state from the requesting 
conference device to said other ones of said conference 
devices enrolled in said conference; and 

processing the application program by said other ones of 
said conference devices enrolled in said conference to 20 
acquire a state equal to the current state. 

2. The conference system as recited in claim 1 wherein the 
application program is a blackboard program for displaying 
a screen on said conference devices on which users can write 
messages. 25 

3. In a conference system including a plurality of confer- 
ence devices wherein a requesting conference device is 
executing an application program, said requesting confer- 
ence device adapted for dynamically sharing the application 
program among the plurality of conference devices while 30 
maintaining said plurality of conference devices in a same 
state, said requesting conference device comprising: 
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application program storage means for storing an appli- 
cation program used for a conference; 

callback protocol storage means, coupled to the applica- 
tion program storage means, for storing a processing 
program along with identifiers for sharing the applica- 
tion program; 

conference control means for maintaining all conference 
devices in a same state coupled to the callback protocol 
storage means, wherein one or more registered identi- 
fiers of the processing program corresponding to event 
information indicating a changed state of a conference 
are previously stored, said conference control means 
communicating the event information corresponding to 
the changed state to all said conference devices, said 
conference control means further for changing a con- 
ference execution state to be equal to the changed state 
corresponding to communicated event information and 
executing the processing program corresponding to the 
communicated event information by referencing the 
registered identifiers; and 

communication control means, coupled to the conference 
control means, for communicating transferred data 
including the event information. 

4. The conference device of claim 3, where the plurality 
of conference devices communicatively couple so that the 
event information can be input or output. 

5. The conference system as recited in claim 3 wherein the 
application program is a blackboard program for displaying 
a screen on said conference devices on which users can write 
messages. 
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[57] ABSTRACT 

The present invention pertains to a system and method for 
managing, a real-time distributed collaborative drawing 
activity. A community of collaborators associated with client 
computers are connected via a communications link with a 
server computer. Each collaborator contributes to the real- 
time design of a shared drawing that is displayed in each 
collaborator's virtual whiteboard. Contributions to the 
shared drawing as well as communications between the 
collaborators are transmitted to the server computer. The 
server computer broadcasts the communication to each of 
the collaborators engaged in the design activity. Each user 
has a set of drawing tools that are used to generate a variety 
of drawing strokes that edit the shared drawing. A curve 
generating procedure is used to capture the manner in which 
the user draws the stroke in order to emulate a selected 
drawing tool. A compact representation of the user's stroke 
is generated and transmitted to each collaborator. 

27 Claims, 13 Drawing Sheets 
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SYSTEM AND METHOD FOR DISTRIBUTED used in drawing the stroke. Time data points and/or pressure 

COLLABORATIVE DRAWING data points can be received from a pointing device used to 

generate the user's stroke. The time and/or data points can 

The present invention relates generally to distributed be used to emulate the drawing tool's natural style. The 

computing systems and particularly to a system and method 5 natural style of a drawing tool can be manifested in the 

for managing a distributed collaborative design activity. varying color hues, texture, and thickness of the stroke. 

BACKGROUND OF THE INVENTION BRIEF DESCRIPTION OF THE DRAWINGS 

Interactive conferencing systems provide users residing in Additional objects and features of the invention will be 

disparate geographic locations the capability to participate in 30 more readily apparent from the following detailed descrip- 

a real time design activity. Typically each user or participant tion and appended claims when taken in conjunction with 

is associated with a computer that is networked with the the drawings, in which: 

computers of the other participants. A display device asso- FIG. 1 is a block diagram of a computer system incor- 

ciated with each user is used as a virtual whiteboard to porating the preferred embodiments of the present invention. 

display and receive drawings and other communications 35 FIG. 2 shows a server computer system according to an 

pertinent to the collaborative design activity. Each partici- embodiment of the present invention. 

pant has the same copy of the shared drawing in its white- , - . , c , 

f j r . • . . * • . FIG. 3 is a schematic representation of an exemplary 

board. Each participant engages in the design activity by •, ,. , . • , . f 

... A. *i , f. *. • . video display screen used in an embodiment of the present 

adding or editing the shared drawing. Alterations to the invention 

shared drawing are quickly transmitted to each participant in 20 t-,.™ ' . . . 

order to maintain a consistent state of the shared drawing 1 FIGS ' 4A ^ D are , schematic representations of menu 

amongst the participants. Thus, the success of such a system selection ltems useti in an embodiment of the present 

relies on an efficient methodology for quickly relaying invention. 

graphical images to each of the participants. 5 * a flow charl of the ste P s used to capture and 

25 transmit drawing alterations made during a collaborative 

SUMMARY OF THE INVENTION design activity. 

The technology of the present invention pertains to an FIGS - 6A-6E are schematic representations of the steps 

apparatus and method for supporting a real-time computer- uscd t0 trace lhe P ath of a curve drawn in a shared drawing, 

networked collaborative design activity. The collaborative FIG. 7 is a flow chart of the steps used to draw a curve in 

design activity includes a community of users that commu- 30 a shared drawing. 

nicate with one another through client computers intercon- FIG. 8 illustrates the steps used by the image generation 

nected via a computer network such as the Internet. In an procedure in displaying an image. 

embodiment of the present invention, the collaborative FIGS. 9 A-9C are schematic representations of steps used 

design activity is a shared drawing activity between a to erase a portion of a shared drawing, 

community of users within the architecture engineering nG 10 ^ a flow chan of lhe st ^ tQ erasc a ion 

construction (AEC) industry. Each user utilizes a video 0 f a snarec j drawing 

display as a virtual whiteboard to engage in the collaborative n a -it • r i 

-i c i_ j j * FIGS. 11A-11C are illustrations of curves used to gen- 

design of a shared drawing. . c n . & 

& & erate a set of parallel lines. 

Each user or collaborator in the collaborative drawing An t ~ .„ , , . , - 

. c . , , - , & 40 FIG. 12 illustrates the steps used to generate a set of 

session has its own copy of the shared drawing. Each user ara ii e l lines 
can edit the shared drawing. Each user's alterations are 

captured and transmitted to the server computer. The server DESCRIPTION OF THE PREFERRED 

computer then broadcasts the alterations to each of the client EMBODIMENTS 

computers associated with a collaborator. Each collaborator 45 

receives the alterations which are then incorporated into System Architecture 

their copy of the shared drawing. ^ technology of lhe present invention pertains to an 

Each collaborator is provided with a set of drawing tools apparatus and method for supporting a real-time computer- 
thai can be used to add or delete lines and curves to the networked collaborative design activity. The collaborative 
shared drawing. Each drawing tool has associated with it a 50 design activity includes a community of users that commu- 
specific style. The present technology captures the manner in n icate with one another through client computers intercon- 
which a user draws a stroke in order to emulate the style of nected via a public computer network such as the Internet, 
the drawing tool used to draw the stroke. The communications between each participant includes all 

A collaborator uses a pointing device to draw a stroke. media forms that can be transmitted via a computer network. 
Location data points are used to represent the path of the 55 Examples of such media can include, but are not limited to, 
stroke. The location data points are then used to generate text, video, sound, and/or graphical images. A server corn- 
curve points representing the curve in accordance with a puter is used to coordinate the transmission of the commu* 
specified resolution. The curve points are used to display an nications between each participant. 

image or the curve. In addition, curve descriptors are gen- In an embodiment of the present invention, the collabo- 

e rated which are a compact representation of the shape of the 60 rative design activity is a shared drawing activity between a 

curve. The curve descriptors are transmitted to the server community of users within the architecture engineering 

computer for broadcast to the other collaborators. A collabo- construction (AEC) industry. Each user utilizes a video 

rator receiving the curve descriptors translates them into display as a shared drawing space that can display one or 

curve points in order to display the image in the collabora- more drawings. These drawings can include computer-aided 

tor's virtual whiteboard. 65 design (CAD) drawings, construction blueprints, archilec- 

1 "he speed and pressure at which portions of the stroke are lural drawings, computer-designed graphic images, and the 

drawn is used to emulate the natural style of the drawing tool like. 
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Each user or collaborator in the collaborative drawing 
session has its own copy of the shared drawing. Each user 
can edit the drawing using one or more graphic devices, such 
as but not limited to, a mouse or other pen pointing device. 
A collaborator can edit the shared drawing by adding a new 
drawing to it, change a part of the existing shared drawing, 
or perform other edit changes. The alterations to the drawing 
by each user are then captured and transmitted to the server 
computer. The server computer then broadcasts the alter- 
ations to each of the client computers associated with a 
collaborator. Each collaborator receives the alterations and 
incorporates them in their copy of the shared drawing. 

In addition, each collaborator can send text messages to 
other collaborators. A dialogue or chat box is provided 
which allows real-time text-based messages to be commu- 
nicated between one or more collaborators during the shared 
drawing session. In other embodiments of the present 
invention, messages including video and audio data can also 
be transmitted between the collaborators. 

Referring to FIG. 1, a system 100 representing a preferred 
embodiment of the present invention includes a number of 
client computers 102 and one or more server computers 104 
in communication via a communications link 106. The 
communication link 106 generically refers to any type of 
wire or wireless link between computers, such as but not 
limited to a local area network, a wide area network, or a 
combination of networks. In a preferred embodiment of the 
present invention, the communications link 106 can be a 
network such as the Internet. 

A client computer 102 includes a central processing unit 
(CPU) 108, a user interface 110, a memory 114, and a 
communications interface 112. The client computer 102 can 
be any type of computing device, such as but not limited to, 
desktop computers, workstations, laptops, and/or mainframe 
computers. One or more users (not shown) can be associated 
with each client computer 102. The communications inter- 
face 112 is used to communicate with other client computers 
102 as well as other system resources not relevant here. 

The user interface 110 can consist of any type and number 
of input/output (I/O) devices, such as but not limited to, a 
video display device, printing devices, a keyboard, a mouse, 
pen pointing devices, graphic tablets (such as a WACOM 
pressure sensitive graphics tablet), and the like. 

The memory 114 of the client computer 102 may be 
implemented as RAM (random access memory) or a com- 
bination of RAM and non-volatile memory such as magnetic 
disk storage. The memory 114 of the client computer 102 
can contain the following: 

an operating system 116; 

network access procedures 118. In an embodiment of the 
present invention, the network access procedures 118 
can be used to implement a communications protocol 
that is suitable for transmitting data through the 
Internet, such as the Transmission Control Protocol and 
the Internet Protocol (TCP/IP); 

an interactive drawing procedure 120 that manages the 
collaborative design activities between several client 
computers 102; 

one or more session transcripts 136. Each session tran- 
script 136 stores a transcription of the events that occur 
during a collaborative design activity. A particular 
session transcript 136 can be replayed at a later point in 
time; 

a drawing repository 138 that stores one or more drawings 
140. A drawing 140 can be stored in any computer- 
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generated image or graphic file formal, such as but not 
limited to BMP, DXF, HPGL, DIB, TGA, GIF, TIF, 
PCX, JPG, and the like; and 
other procedures and data structures. 

The interactive drawing procedure 120 can contain the 

following: 

one or more curve engine procedures 122. Each curve 
engine procedure 122 is used to generate the curve 
descriptors and curve data points used to represent a 
curve; 

image generation procedure 124 that display images onto 
any type of I/O device, such as a video display device; 

session transcript procedures 126 that record portions of 
or the entire design activity session into a session 
transcript 136. In addition, the session transcript pro- 
cedures 126 enable a user to replay a particular session 
transcript 136 at a later point in time; 

conferencing procedures 128 that manage the exchange of 
communications between each collaborator during a 
design activity session; 

conversion procedures 130 that convert an incoming 
drawing stored in a particular file format into a format 
suitable for use by the interactive drawing procedure 
120; 

menu display procedures 132 that manage the drawing 
facilities provided to a user; and 

distance mapping procedures 134 that map a lime and/or 
pressure value into an equivalent measure of distance; 

as well as other procedures and data structures. 
Referring to FIG. 2, there is shown a server computer 104 
including a central processing unit (CPU) 202, a user 
interface 204, a memory 208, and a communications inter- 
face 206. The memory 208 of the server computer 104 can 
contain the following: 

network access procedures 210. In an embodiment of the 
present invention, the network access procedures 210 
can be used to implement a communications protocol 
that is suitable for transmitting data through the 
Internet, such as the Transmission Control Protocol and 
the Internet Protocol (TCP/IP); 

an operating system 212; and 

other procedures and data structures. 

The operating system 212 of the server computer 104 
utilizes broadcasting procedures 214 that manage the con- 
ferencing activities between the various client computers 
102 engaged in the collaborative design activity. In an 
embodiment of the present technology, the UNIX operating 
system is used which supports a multi-user backbone 
(M-bone) or network ability that allows multiple client 
computers 102 to connect to a particular server computer 
104. However, it should be noted that the present invention 
is not limited to this particular type of operating system or 
broadcasting facility. Others can be used so long as they 
provide the same functionality. 

The server's operating system procedures 212 utilizes a 
network port to support a number of communication streams 
that are used to transmit data between the client computers 
102 engaged in the collaborative design activity. The serv- 
er's operating system 212 receives transmissions from one 
or more client computers 102 and routes them to the 
intended client computers 102. Initially, a client computer 
102 sends a request to the server computer 104 to connect. 
The request contains the Internet address of each client 
computer 102 of a collaborator. The server computer 104 
makes the connection and establishes a stream representing 
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a connection between the client computer 102 with its In some instances, time and pressure data points are 

network port. This stream allows communications to be received along with the position data points. The time data 

transmitted rather rapidly between the server computer 104 points reflect the speed at which the stroke was generated 

and the particular client computer 102. anc j , ne pressure data points reflect the pressure used to 

Each collaborator participates in a collaborative drawing 5 generate the stroke. The time and pressure points are used to 

activity by utilizing the video display as a "virtual" white- capture the natural sl , e of the drawing t00 , used l0 draw the 

board.. The whiteboard can display any and all graphic stroke naturai sl le of a ^ wing tool ^ manifested by 

images that are part of the collaborative drawing activity. ^ . coJor faues k ^ slfok b me width of |hc 

FIG. 3 illustrates an exemplary screen display in accordance strok and b ^ f ^ Fo / instancC) Qts 

with an embodiment of the present invention. The mterac- * * \ % J r . + ^ L 

live drawing procedure 120 displays the virtual whiteboard 10 of a Strc * e drawn fast *f e re P re f nted b J a da^r taw or 

302 along with a menu bar 304, a dialogue box 306, and a texturc tha " ^S" 3 ™* thal are **wn slower. In addition, 

text input box 308. The virtual whiteboard 302 can consist segments of a stroke that are drawn fast are often thinner 

of one window illustrating a particular drawing 310, can than lhose drawn sIowl y- Likewise, the pressure of a stroke 

include multiple windows (not shown) with each window can affect the color hue » texture, and width characteristics, 

focusing on .a particular aspect of the drawing 310, or 15 For example, segments of a stroke that are generated with 

multiple windows (not shown) with each window displaying heav y pressure can be reflected with a darker hue, darker 

a different drawing. The dialogue box 306 is used to display texture and with a wider shape. By contrast, a lightly drawn 

text messages that are transmitted and received during a stroke can be reflected with a lighter hue, lighter texture and 

collaborative drawing session. The text input box 308 is with a dinner shape. 

used by a user to insert text messages that are distributed via 20 The characteristics of each drawing tool varies. As such, 

the server computer 104 to the other collaborators of the the time and pressure data points are used differently to 

collaborative drawing session. reflect the natural style of a particular drawing tool. For 

Each collaborator can utilize a different type of video example, pressure has an effect on the stroke that is gener- 

display device. For this reason, each collaborator's white- ated by a water color brush but speed may not. The pressure 

board is presumed to be of a standard size and is scaled, if 25 used to draw a stroke with a water color brush will affect the 

necessary, to accommodate the dimensions of a particular width of the stroke but not the hue. By contrast, pressure and 

collaborator's video display device. speed affect the style of a wet paint brush stroke. The more 

Each collaborator can edit the drawing in a variety of pressure that is used to draw the stroke, the thicker and 

ways using different tools 312 to draw different objects, darker the stroke becomes. Similarly, the faster the stroke is 

lines, strokes, shapes, and the like. A menu display proce- 30 drawn, the thicker and darker the stroke becomes, 

dure 132 provides a number of facilities that enable the user The curve generating method of the present technology 

to edit a drawing in an intended manner. FIGS. 4A-4D traces the path of a user's stroke and generates a curve from 

illustrate an exemplary list of such facilities. For instance, it. The curve is represented as a set of curve points and as a 

the user can select a particular drawing tool 312 from a menu set of curve descriptors. The curve points are used to display 

that includes, but is not limited to, the following tools: a fine 35 the image in accordance with user-defined directives. The 

pen, a wet brush, a watercolor brush, a pencil, a charcoal user-defined directives can specify the curve's resolution 

pen, an etch device, a straight edge, a crayon, a piece of a nd the curve characteristics relative to a particular drawing 

chalk and/or a rollover. A user can use a particular tool to tool. 

draw a stroke or text in its whiteboard. In addition, one or ^ curve descriptors define tne shape of lhe curve . 

more templates of predefined shapes 314 or objects is curve descriptors are ^ t0 repr esent a curve: a first 

provided for the user. The style of each stroke, shape, or text descriptor represe nis a starting point of the curve; a second 

can be associated with one or more styles 318 and/or descr i p ,or represents the shape of the curve; and a third 

textures 316. The textures 316 can represent various fill descrip , or represents an ending point of the curve. The curve 

patterns. In an embodiment of the present invention, the descriptors are transmitted to the collaborators of a collabo- 

textures 316 can represent materials used m the AEC indus- «* ra(ive dt ^ ^ descri , ors afe transmitted 

try. For instance, a diagonal cross-hatch texture can repre- ratber (nan , he ition data ints ta order t0 mimrni2e , he 

sent concrete, a orthogonal cross-hatch texture can represent amoun , of dala lna , ^ transmitted belween the collaborators, 

ceramic title, and so forth. In this manner, the speed oftbe transmission is increased and 

The general architecture and processing associated with relatively low bandwidth transmission links can be utilized 

the interactive collaborative design activity has now been 50 wjthou( com isjng (he speed „ whkh drawi u ^ a(es 

disclosed. Attention presently turns to the curve generating are broadcast 

techniques associated with the present technology. . . . . . , 

r °' 1 ne curve generating method of the present technology 

Curve Generating Method differs from curve fitting techniques. Curve fitting tech- 

In the collaborative drawing session of the present 55 niques "fit" or generate a curve from a set of user-defined 

invention, each user can edit the shared drawing. Typically, control points that describe the shape of the curve. The 

a user edits the drawing by adding or deleting strokes to the control points arc not necessarily pan of the curve. By 

shared drawing. A pointing device is used to trace the path contrast, the present technology receives a curve and trans- 

of the user's stroke. The pointing device represents the path Iates > l ml ° another curve in accordance with one or more 

as a number of position data points. The position data points 60 user-defined directives (i.e., resolution value and drawing 

are then mapped into curve descriptors which are a concise tool). 

representation of the Bezier curves that form the stroke. In Prior to discussing the details of the curve generating 

addition, the stroke is also represented by a number of curve methods, a brief synopsis of the method is described first 

points that are used by the image generation procedure to followed by a glossary of terms used in this document 

display the stroke. I Tie curve descriptors are transmitted to 65 concerning the curve generating method. The curve gener- 

the server computer and broadcasted to the other collabo- ating method utilizes Bezier curve methods which are well- 

rators. known in the art. A more detailed description of the Bezier 
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curve methods can be found in Newman and Sproull, 

Principles of Interactive Computer Graphics, 2d edition, qdm)- ^i(0)fl2,o(«*)- v l (2)B 2 aM (2) 

McGraw-Hil] Book Company, which is hereby incorporated l ^ = 5 2 ,i(u*) 
by reference as background information. 

The curve generating method maps position data points 5 where u k is the interval that represents the furthest distance 

received from a pointing device into a set of curve data from the starting and ending points and can be determined 

points that define one or more Bezier curves. The curve via the following mathematical relation: 
engine 122 performs this mapping in real time and is 

sensitive to real-lime processing constraints. 10 .-^{Iw^il+^r^l} 0) 

Astroke is represented as a series of second degree Bezier where x 0 ,y 0 is the starting point of the curve segment, x„,y„ 

curves. Each Bezier curve is represented by a mathematical is the ending point of the curve segment, and x^y, are all 

function q(u). The function q(u) is defined over a sat of other location curve points in the curve segment, 

evenly-distributed intervals represented by the vector u. Stroke: The curve drawn by a user. Astroke is represented 

Each interval u ; is a real number within the range between 35 by a series of curve segments q x 3 (u), q 3 2 (u), . . . , q a „(u). 

0 to 1. The number of intervals, n, effects the resolution of Location curve descriptors: Control points that define a 

the curve. The number of intervals can be either a user- curve segment and are defined above, 

defined value or determined from the curve engine 122. In Time curve: A series of time segments representing a time 

order to process the position data points in real time, the dimension associated with the user drawn stroke, 

curve engine 122 may adjust the number of intervals in order 20 Time segment: A second degree Bezier curve, q 2 (u), that 

to meet the demands of real-time processing. is represented by a vector, u, of time curve points, u y , that are 

Time and pressure data points can also be received from defined via the mathematical relation: 
the pointing device. These data points are also transformed 

into a series of second degree Bezier curves in accordance 2 5 ^ * 4) 

with the same mathematical function q(u). The time and tt(«,0 = ^Vi(O*rf,.-(«/) 
pressure Bezier curves are used by the image generation 
procedure 124 to generate an intended image of the curve. 

To assist the reader, the following glossary of terms is where 

used to describe the curve generating method: 30 ' 

u y - represents a relative location along the curve. The 

Position data point: An (x, y) integer pair representing the values of u are real numbers u„ such that 0^u„^ 1, where n 

logical location of the device pointer relative to the upper controls the resolution of the curve, 

left hand corner of the video display device. B rf ^u i ; {l-u ; ) </ " 1 , and 

Time data point: An integer value within the range [0, 35 V 2 t (0 a ™ time curve descriptors. V 2 (0) represents a start- 

65535] that represents a time unit at which a corresponding in S time P oint » V 2< 2 ) represents an ending time point, 

position data point is generated. In an embodiment of the V 2(!) represents a shape descriptor represented in 

present invention, the time unit is expressed in milliseconds accordance with the following mathematical relation: 
and represents the time differential from the immediately 

previous point. 40 _ ^) - ^2(0)Bi fi (u k ) - v 2 {2)B 2i2 {u h ) (5) 

Pressure data point: An integer value within the range 2,1 ( ^ 
[0,255] that represents the pressure associated with a cor- 
responding position data point. In an embodiment of the where u* is the interval that represents the furthest distance 
present technology, 0 represents no pressure and 255 rep- 45 from the starting and ending points and can be determined 
resents the heaviest amount of pressure. via the following mathematical relation: 

Curve segment: A second degree Bezier curve, qj(u), that 

is represented by a vector, u, of location curve points, u ; , that max o « y< „{[x o ,> 0 -.t /I > f 1+1x^-^^1} (6) 

are defined via the mathematical relation: where x 0 ,y 0 is the starting time point, x„,y„ is the ending 

time point, and x,-,y t - arc all other time curve points in the 

' U) curve segment. 

7iiu;) = 2j u il0%(«;) Pressure curve: Aseries of pressure segments representing 

a pressure dimension associated with the user drawn stroke. 

55 Pressure segment: A second degree Bezier curve, q^u), 

wnere that is represented by a vector, u, of pressure curve points, 

u,, that are defined via the mathematical relation: 

d-2, 

Uy represents a relative location along the curve. The * (i) 

values of u are real numbers u M such that 0^u M ^l, 6Q 93(«;)= ^ VjW^fu,-) 

where n controls the resolution of the curve, I *° 
B^u'/l-u/', and 

V 1 (i) are location curve descriptors. V^O) represents a where 

starting point of the curve segment, Vj(2) represents an d=2, 

ending point of the curve segment, Vj(l) represents a 65 u- represents a relative location along the curve. The 

shape descriptor represented in accordance with the values of u are real numbers u„ such that 0^u„=il, 

following mathematical relation: where n controls the resolution of the curve, 
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^/■u'^l-Uy) and when the direction of the angle 0 is opposite of the direction 

V 3 (i) are pressure curve descriptors. V 3 (0) represents a of tne previously-calculated angle. 

starting pressure curve point, V 3 (2) represents an end- A third curve descriptor V(l) is generated to represent the 

ing pressure curve point, V 3 (l) represents a shape shape or peak of the curve segment in accordance with 

descriptor represented in accordance with the following 5 equation (2) above. FIG. 6C illustrates the various curve 

mathematical relation: segments formed from a user's stroke and their respective 

curve descriptors. The curve descriptors for each curve 

<?3(«a) - Vjto^otw*) - Vi{2)B 2 jUt t ) (3) segment of a user's stroke is transmitted to the collaborators 

V *M = B u {u k ) engaged in the drawing activity. 

10 A Bezier curve is formulated for each defined curve 

segment. The Bezier curve is represented as a series of curve 

where u* is the interval that represents the furthest distance points. A curve segment is displayed as a series of lines that 

from the starting and ending points and can be determined connect the curve points as shown in FIG. 6D. Thus, the 

via the following mathematical relation: number of curve points determines the resolution of the 

15 curve. The curve points are transmitted to the image gen- 

n , , ,„ eration procedure 124 which displays the resulting curve as 

-^.{Iw^fco^JI (9) muslra £, - m mG 6E § 

where x 0 ,y 0 is the starling pressure curve point, x„,y„ is the In addition, time and pressure data points can be received 

ending pressure curve point, and x„y, are all other pressure from the pointing device along with the position data points, 

curve points in the curve segment. 20 A time data point can be an integer pair representing a time 

FIG. 5 illustrates the steps used in the curve generating value and an index. The time value can be within the range 

method of an embodiment of the present invention. During [0, 65535] and is expressed in milliseconds. The index 

a collaborative drawing session, a collaborator can draw a represents the order of a time data point relative to other time 

stroke utilizing a pointing device. The pointing device data points. The time data points are mapped into a set of 

represents the locations pointed to by the user as a set of 25 time curve points and lime curve descriptors. The time curve 

position data points. The position data points are integer points are used along with the location data points to display 

value pairs (x,y) that represent the logical location of the the image in accordance with the natural style of a particular 

device pointer relative to the upper left hand corner of the drawing tool. The time curve descriptors are transmitted to 

video device. The curve generation procedure 122 receives other collaborators along with the location curve descriptors 

the position data points in real-time and simultaneously 30 and/or pressure curve descriptors in order to display the 

generates curve points and curve descriptors (step 502). The curve in the collaborator's whiteboard, 

details of the generation of the curve points and curve Similarly, a pressure data point can be an integer pair 

descriptors will be discussed below. representing a pressure value and an index. The pressure 

The curve points are then used by the image generation data point value can be within the range [0, 255]. The index 

procedure 124 to display the curve on the user's video 35 represents the order of a pressure point relative to other 

display device or other I/O device (step 504). The curve pressure points. The pressure data point is mapped into a set 

descriptors are formatted and transmitted to the server of pressure curve points and pressure curve descriptors. The 

computer 104 for broadcast to other collaborators (step 506). pressure curve points are used along with the location curve 

The server computer 104 receives this data and broadcasts it points and/or time curve points to display the image in 

to the intended collaborators (step 508). A collaborator 40 accordance with the natural style of a particular drawing 

receives the transmission, generates the appropriate curve tool. The pressure curve descriptors are transmitted to other 

points that correspond to the curve descriptors, and displays collaborators along with the location and/or time curve 

the new image (step 510). descriptors in order to display the curve in the collaborator's 

FIGS. 6A-6E illustrate the curve generating method. The whiteboard, 

user's stroke is represented by a series of position data points 45 FIG. 7 illustrates the steps used in generating a curve in 

that are represented as x,y integer pairs. The position data an embodiment of the present technology. It should be noted 

points are partitioned into curve segments and a Bezier that a user's stroke can include lines as well as curves and 

curve is defined for each curve segment. The Bezier curve that the curve generation engine 122 can accommodate lines 

consists of a number of curve points which are used to as well as curves. The following discussion focuses on 

display the resulting image. The number of curve points is 50 curves and can be easily modified by one skilled in the art 

a funciion of the curve's resolution. Coarsely drawn curves to accommodate lines. 

are represented by a smaller number of curve points than Further, the following discussion pertains to an erabodi- 
smoothly drawn curves. ment where the pointing device retrieves location, time, and 
Referring to FIG. 6 A, each integer pair (x,y) is received pressure data points from a user's stroke. In alternate 
in real time by the curve engine 122. The first point (x 0 ,y 0 ) 55 embodiments, the pointing device would retrieve any corn- 
is denoted as the starting curve point of the first curve bination of location data points and either time or pressure 
segment. The starting point is also the first curve descriptor data points. One skilled in the art would be able to easily 
V(0). Each subsequent point is analyzed to determine modify the discussed methodology to accommodate these 
whether it is part of the first curve segment. Referring to alternate embodiments. In yet another embodiment, the time 
FIGS. 6B-6C, this is doe by determining the angle, 0, 60 data points need not be extracted from the pointing device, 
between the starting point, (xQ,y 0 ), and the previous data The time data points can be calculated by the curve genera- 
point (Xj,yj) and the starting point, (x 0 ,y 0 ), and the current tion engine 122 using the location data points. This can be 
data point (x 2 ,y 2 ). If the angle 6 exceeds a predetermined performed by calculating the distance between two succes- 
threshold, the previous data point (x^yj is made the ending sive location data points and mapping the distance into an 
point V(2) of one curve segment and the starting point V(0) 65 appropriate time unit. In addition, other methodologies can 
of a new curve segment. The ending point V(2) is also a be used to calculate time data points from the location data 
curve descriptor. In addition, a new curve segment is formed points. 
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Referring to FIG. 7, the curve engine 122 receives in real Otherwise (step 532-N), the appropriate curve descriptors 
time a number of data points (step 520) represented as and curve points are calculated (step 536). The Vj(l), V 2 (l), 
follows: V 3 (l) curve descriptors are calculated as described in equa- 
ls location data point represented as (x,y). lions (2), (5), and (8) and the curve points q^u), q 2 (u), and 
t,: time data point represented as (z„ zjwhere z a is an 5 q 3 (u) are calculated as well as described in equat ions (1), (4), 
integer value within the range [0,65535] and a time unit ' 

expressed in milliseconds and where z, represents the Sle P s 52a " 536 are re P eated unlil no more dala P° mts arc 

index of the time data point relative to other time data received b * thc CUfve eD & ine Procedure 122 from the point- 

p 0 | nls ing device. This typically signifies the fact that the user has 

, A - y x ■ . io completed the stroke by removing the pointing device from 

Pl , pressure data po.m represented as (p l P2 ) where Pl is the visual display device . 

an integer value within the range [0, 255] and where p, n c . t J. ir , _ . . r . , 

& t 4 . . , ~. , . ■ . i „■ Referring to FIG. 5, upon the completion of the above 

represents the index of the pressure date point relative 4 . , & . r lt _ , * , 

. r j . ■ . mentioned curve generating method, the curve descriptors 

to other pressure data points. . ... . . & . to . - , , . . r , 

r r are transmitted to the server computer for broadcast to the 

Vi/0), Vj/1), V at( <2): curve descriptors for curve seg- 35 othef collaborators (step 506). The transmission can be 

menl J* formatted to include the following: drawing tool indicator, 

v 2y<°). v ^O)» V 2/ 2 ) : curve descriptors for time curve location curve descriptors, time curve descriptors, and pres- 

segment j. sure curve descriptors. The drawing tool indicator specifies 

V 3; (0), V 3i (l), V 3/ <2): curve descriptors for pressure curve the particular drawing tool used to make the stroke, 

segment j. 20 When a collaborator receives the curve descriptors, the 

The curve engine procedure 122 determines if the received curve engine 122 transforms thc curve descriptors into a 

dala points are the first set of data points (step 522), In this respective set of curve points in accordance with equations 

case (step 522- Y), the start and end curve descriptors V(0) (1), (4), and (7). The curve points are then transmitted to the 

and V(2) for each data point type is initialized as follows image generation procedure 124 which displays the curve in 

(step 524); 25 the collaborator's whiteboard. 



VyCOyV^)-/,,- V,,(0)- V2)-f /; V 3/ (0)-V 3/ (2)o A , (10) 



Image Generating Method 



V„{ 2 W ( -; W-V V 3 ;(2)=p, (31) 



Referring to FIG. 8, the image generation procedure 124 
If the received data points are the second set of data points 3Q can reC eive three sets of curve points: a set of location curve 
(step 526), the end curve descriptors are set to the respective pomls qj ^ qi ^ a ^ 0 f time curve points q 2 j(u) . . . 
second data point as follows (step 528): q 2 ^,(u); and a set of pressure curve points q 3fl (u) , . q 3 ^(u) 

which it uses to display the associated image 556. The image 
generation procedure 124 can utilize a scale procedure 550 
to map the location curve points q i j(u) . . . q 1>B (u) into 
For each subsequent set of data points, the direction of the suitable coordinates for the particular collaborator's visual 
curve and its growth is determined relative to the starling display device. In addition, each drawing tool has associated 
point or first curve descriptor (step 530). This is performed with it one or more scale tables 552, 554 that are used to map 
by determining the angle between two secant lines repre- the time curve points q 21 (u) . . . q 2j/1 (u) and pressure curve 
senting the previous location data point and the current 4Q points q 34 (u) . . . q 3> „(u) into suitable color and dimension 
location data point (see FIG. 6B). This can be determined attributes associated with the video display device, 
mathematically as follows: Th e general curve and image generating methods associ- 

ated with the collaborative design activity have now been 
a , f/\ ii i\ a II h v disclosed. The present technology provides two additional 

0-arctan ((^,-y 0 -br-^-Cki-^-l^i-JtoD) C 12 ) ...... T n u a- a e a 11 .u 

45 capabilities that will now be discussed. Specifically the 

In addition the direction of the angle G is determined and capability for erasing a displayed curve and the capability of 

compared with the direction of the immediately preceding automatically generating parallel lines from a single user's 

calculated angle. Thc direction of the angle 8 is expressed as stroke, 
being either a clockwise direction or as a counter clockwise 

direction. 50 Erasure 

The computed angle 9 and its direction is used to deter- 
mine if the received location data point is past a critical point Durin S a collaborative drawing session, a user can erase 
defining the current curve segment (step 532). This is a °y lin es or ™™ s m the shared drawing. For illustration 
determined by testing the computed angle 6 against a purposes, the situation where a user specifies two points 
user-defined threshold. In an embodiment of the present 55 indicating where a curve is to be erased will be described, 
technology this threshold is between the range (:t/4, ji/10). However, it should be noted that the present technology can 
If the direction of the angle is opposite to the previously erase a iine or m an y number of locations, 
calculated angle, this will indicate that the received location Referring to FIGS. 9-10, a user indicates with the point- 
data point is outside the realm of the current curve segment. ing device two points 560, 562 on the curve that define an 
If it is determined that the received data points define a 60 area of the curve that is to be erased. These points 560, 562 
new curve segment (step 532-Y), the curve descriptors V(0) are received by the curve generating procedure 122 (step 
and V(2) are initialized to the respective data points as 570). The curve generating procedure 122 determines the 
follows (step 534): effected curve segments and their associated curve descrip- 
tors (step 572). For the curve segment corresponding to the 
65 first erasure data point (step 574- Y), the curve segment 's end 
viv-r- i, >}*i\ t i> j^iv*-/ point is adjusted to be first erasure data point. Likewise, for 
v tftl (0H w ; v z,>i(0>W v >j»i(0hp.--i- 03) the curve segment corresponding to the second erasure dala 
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point (step 574-N), the curve segment's starting point is 
adjusted to be the second erasure data point (step 578). The 
appropriate curve data points qj(u), q 2 (u), q 3 (u) representing 
the affected curve segments are recalculated as well as the 
V(l) curve descriptors in accordance with the aforenaen- 5 
tioned equations (step 580) (see FIG. 9B). Also, any curve 
segments located entirely between the deletion points 560, 
562 are deleted (step 580). 

The curve data points are then transmitted to the image 
generation procedure 124 to display the new image (see FIG. ]Q 
9C). The curve descriptors for both of the newly generated 
curves are transmitted to the server computer 104 for 
broadcast to each collaborator. 

Parallel Curves 

Another facility of the present technology is the capability 
to automatically generate parallel lines or curves from a 
user's stroke. For illustration purposes, the following dis- 
cussion will focus on the generation of two parallel curves. 
However, one skilled in the art can easily modify the 
described technique to generate parallel lines and multiple 20 
parallel lines and curves. 

A user traces the path of a curve with a pointing device. 
The curve is represented as a series of position data points. 
Each of the two parallel curves will be generated at an equal 
distance above and below the user's stroke. Time and/or 25 
pressure data points associated with the drawn curve are 
used to determine the distance between the parallel curves 
(see FIG. 11 A). In an alternate embodiment, the time data 
points need not be received from the pointing device. The 
time data points can be inferentially calculated by measuring 30 
the distance between two position data points as previously 
described above. In addition, the time and/or pressure data 
points can also be used to affect the width, texture, and color 
hue of each of the parallel curves as previously described 
above. 35 

Referring to FIG. 12, a set of position data points (x,-,y ; ) 
and time and/or pressure data points are received from an 
I/O device 582. The time and/or pressure data points are 
mapped via a distance mapping procedure 134 into an 
appropriate distance measure d ( . The distance is divided in 4 0 
half. The distance (d/2) and the position data points are 
transmitted simultaneously to two curve engines 122. The 
first curve engine 122a generates the curve points and curve 
descriptors for the top parallel curve and the second curve 
engine 1226 generates the curve points and curve descriptors *5 
for the bottom parallel curve. The curve points for both 
curves are then transmitted to the image generation proce- 
dure 124 which displays the parallel curves 584, 586 as 
shown in FIG. 12B. The curve descriptors are transmitted to 
the server computer 104 for broadcast to the other collabo- 50 
rators. 

In an alternate embodiment, a user's stroke can be 
mapped into an image including two parallel curves where 
the area between the parallel curves includes a user-defined 
texture as shown in FIG. 11 C. The above mentioned steps 55 
for generating parallel curves is used to generate two parallel 
curves. Next, a user-defined texture is drawn in the area 
between the curves. This texture can also be affected by the 
time and pressure data points associated with the user drawn 
stroke. Curve descriptors are computed for this set of 60 
parallel curves and are transmitted along with the texture 
data to the server computer 104 for broadcast to the other 
collaborators. 

Alternate Embodiments 

While the present invention has been described with 
reference to a few specific embodiments, the description is 



,082 

14 

illustrative of the invention and is not to be construed as 
limiting the invention. Various modifications may occur to 
those skilled in the art without departing from the true spirit 
and scope of the invention as defined by the appended 
claims. 

The present invention is not limited to the computer 
system described in reference to FIG. 1. It may be practiced 
without the specific details and may be implemented in 
various configurations, or makes or models of distributed 
computing systems, tightly-coupled processors or in various 
configurations of loosely-coupled microprocessor systems. 

Further, the method and system described hereinabove is 
amenable for execution on various types of executable 
mediums other than a memory device such as a random 
access memory. Other types of executable mediums can be 
used, such as but not limited to, a computer readable storage 
medium which can be any memory device, compact disc, or 
floppy disk. 

The present technology has been described with reference 
to curves and lines. However, the present technology is 
amenable to any two-dimensional graphic image. 
Furthermore, one skilled in the art can modify the above 
discussed technology to accommodate any n-dimensional 
graphic image. 

What is claimed is: 

1. A method for generating a drawing in a distributed 
computing system including at least two client computers 
and at least one server computer, the method comprising the 
steps of: 

(a) providing one or more drawing tools for use by users 
associated with the client computers to draw a curve, 
each drawing tool having one or more style attributes 
associated therewith; 

(b) at a first one of the client computers, receiving a 
plurality of location data points representing a path of 
a curve drawn with one of the drawing tools; 

(c) at the first client computer, mapping the location data 
points into a plurality of location curve data points, the 
location curve data points used to display the curve in 
accordance with a specified image resolution; 

(d) utilizing the location curve data points to display the 
curve in accordance with the drawing tool used to draw 
the curve; 

(e) at the first client computer, generating one or more sets 
of location curve descriptors, the location curve 
descriptors representing a compact representation of 
the location curve data points; and 

(f) transmitting the location curve descriptors from the 
first client computer to the server computer for retrans- 
mission to at least a second one of the client computers; 

(g) at the second client computer, converting the location 
curve descriptors into a plurality of location curve data 
points, the location curve data points used to display the 
curve in accordance with a specified resolution and in 
accordance with a specified drawing tool; 

wherein each step performed at the first client computer is 
also performed at the second client computer and each 
step performed at the second client computer is also 
performed at the first client computer so as to enable the 
users at the first and second client computers to coop- 
eratively generate the drawing. 

2. A method for generating a curve in a distributed 
computing system including at least one client computer and 
at least one server computer, the method comprising the 
steps of: 
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(a) providing one or more drawing tools for use by users 
associated with the client computers to draw a curve, 
each drawing tool having one or more style attributes 
associated therewith; 

(b) receiving a plurality of location data points represent- 
ing a path of a curve drawn with one of the drawing 
tools; 

(c) mapping the location data points into a plurality of 
location curve data points, the location curve data 
points used to display the curve in accordance with a 
specified image resolution; and 

(d) utilizing the location curve data points to display the 
curve in accordance with the style attributes of the 
drawing tool used to draw the curve; 

step (b) further including: 

receiving a plurality of temporal data points represent- 
ing a speed at which portions of the curve was 
drawn; 
step (c) further including: 

mapping the temporal data points into a plurality of 
temporal curve points, each of the temporal curve 
points associated with a corresponding curve data 
point; and 
step (d) further including: 

utilizing the temporal data points to display the curve in 
accordance with the drawing tool used to draw the 
curve. 

3. The method of claim 2, wherein the temporal data 
points are calculated from a distance between consecutive 
location data points. 

4. A method for generating a curve in a distributed 
computing system including at least one client computer and 
at least one server computer, the method comprising the 
steps of: 

(a) providing one or more drawing tools for use by users 
associated with the client computers to draw a curve, 
each drawing tool having one or more style attributes 
associated therewith; 

(b) receiving a plurality of location data points represent- 
ing a path of a curve drawn with one of the drawing 
tools; 

(c) mapping the location data points into a plurality of 
location curve data points, the location curve data 
points used to display the curve in accordance with a 
specified image resolution; and 

(d) utilizing the location curve data points to display the 
curve in accordance with the drawing tool used to draw 
the curve; 

step (b) further including: 

receiving a plurality of pressure data points represent- 
ing a pressure at which portions of the curve was 
drawn; 
step (c) further including: 

mapping the pressure data points into a plurality of 
pressure curve points, each pressure curve point 
associated with a corresponding curve data point; 
and 

step (d) further including: 

utilizing the pressure data points to display the curve in 
accordance with the drawing tool used to draw the 
curve. 

5. The method of claim 1, 

the style attributes including width attributes and color 
attributes. 
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6. The method of claim 2, 

generating one or more sets of temporal curve descriptors, 
the temporal curve descriptors representing a compact 
representation of the temporal curve data points; and 

transmitting the temporal curve descriptors to the server 
computer for broadcast to one or more of the client 
computers. 

7. The method of claim 6, 

for each of the receiving client computers, converting the 
temporal curve descriptors into a plurality of temporal 
curve data points, the temporal curve data points used 
to display the curve in accordance with a specified 
resolution and in accordance with a specified drawing 
tool 

8. The method of claim 4, 

generating one or more sets of pressure curve descriptors, 
the pressure curve descriptors representing a compact 
representation of the pressure curve data points; and 

transmitting the pressure curve descriptors to the server 
computer for broadcast to one or more of the client 
computers. 

9. The method of claim 8, 

for each of the receiving client computers, converting the 
pressure curve descriptors into a plurality of pressure 
curve data points, the pressure curve data points used to 
display the curve in accordance with a specified reso- 
lution and in accordance with a specified drawing tool. 

10. An interactive drawing apparatus, comprising: 

one or more client computers, each said client computer 
including: 

a memory for storing a graphical drawing; 
an input mechanism for receiving one or more hand- 
drawn shapes applied to said graphical drawing; 
at least one curve engine for partitioning each said 
hand-drawn shape into a plurality of adjacent curves, 
each said adjacent curve having at least one 
boundary, each said boundary determined based on 
changes in direction of said hand-drawn shape meet- 
ing predefined threshold criteria, said curve engine 
representing each said adjacent curve as a set of 
control points defining said curve; and 
at least one server computer, coupled by one or more 
communication links to the client computers, the at 
least one server computer including a communication 
mechanism for receiving said control points and trans- 
mitting said control points to one or more of said client 
computers. 

11. The apparatus of claim 10, 

said curve engine generating a set of location curve points 
associated with said control points, said set of location 
curve points used to display said hand-drawn shape in 
accordance with a predetermined criteria; 

each said client computer including: 

an output device for displaying said graphical drawing; 
and 

an image generation procedure that receives said loca- 
tion curve points and displays said associated hand- 
drawn shape in accordance with said location curve 
points as part of said graphical drawing on said 
output device. 

12. The apparatus of claim 10, 
each said client computer including: 

a conferencing procedure for managing an exchange of 
communications between one or more of said client 
computers. 
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13. The apparatus of claim 10, 
each said client computer including: 

a conversion procedure that imports a received drawing 
associated with a first format into said drawing 
associated with a second format. 5 

14. The apparatus of claim 10, 
each said client computer including: 

a session transcript procedure that records alterations 
made to said drawing by each of said client com- 5 q 
puters. 

15. A computer program product for use in conjunction 
with a computer system, the computer program product 
comprising a computer readable storage medium and a 
computer program mechanism embedded therein, the com- 15 
puter program mechanism comprising: 

a graphical drawing; 

an input mechanism for receiving one or more hand- 
drawn shapes applied to said graphical drawing; 2Q 

at least one curve engine for partitioning each said hand- 
drawn shape into a plurality of adjacent curves, each 
said adjacent curve having at least one boundary, each 
said boundary determined based on changes in direc- 
tion of said hand-drawn shape meeting predefined 25 
threshold criteria, said curve engine representing each 
said adjacent curve as a set of control points defining 
said curve; and 

a broadcast mechanism for transmitting said control 
points to one or more of said client computers. 30 

16. The computer program product of claim 15, 

said curve engine generating a set of location curve points 
associated with said control points, said set of location 
curve points used to display said hand-drawn shape in 
accordance with a predetermined criteria; 35 

an output device for displaying said graphical drawing; 
and 

an image generation procedure that applies said location 
curve points to said graphical drawing on said output 40 
device. 

17. The computer program product of claim 15, 

a conferencing procedure for managing an exchange of 
communications between one or more of said client 
computers. 45 

18. The computer program product of claim 15, 

a conversion procedure that converts a received drawing 
associated with a first format into a second format 
associated with said graphical drawing. 

19. The computer program product of claim 15, 50 
a session transcript procedure that records alterations 

made U> said graphical drawing by each of said client 
computers. 

20. A computer-implemented method for generating and 
displaying a curved image, said method comprising the steps 55 
of: 

(a) receiving a hand-drawn shape of a curved image; 

(b) obtaining a distance measurement d from said hand- 
drawn shape; 60 

(c) providing a set of parallel curved images, each of said 
parallel curved images represented as a plurality of 
adjacent curves, each said adjacent curve having at 
least one boundary, each said boundary determined 
based on changes in direction of said hand-drawn shape 65 
meeting a predefined threshold, a first one of said 
parallel curved images located at a distance d/2 above 
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said shape and a second one of said parallel curved 
images located at a distance d/2 below said shape; and 
(d) displaying each of said parallel curved images. 

21. The method of claim 20, 

wherein said hand-drawn shape is received as a plurality 

of position data points; 
said step (b) further comprising the steps of: 

(i) obtaining a plurality of temporal data points, each 
said temporal data point associated with a respective 
one of said position data points; and 

(ii) mapping each of said temporal data points into a 
corresponding distance data point, said distance data 
point including a specified location for positioning 
an associated position data point; 

said step (c) further comprising the step of: 

(i) utilizing said position data points and said distance 
data points to generate said set of parallel curved 
images. 

22. The method of claim 21, 

said step (b)(i) further comprising the step of: 

computing a temporal data point for each of said 
position data points, each said temporal data point 
representing a distance between a current position 
data point and an immediately preceding position 
data point. 

23. The method of claim 20, 

wherein said hand-drawn shape is received as a plurality 

of position data points; 
said step (b) further comprising the steps of: 

(i) receiving a plurality of pressure data points, each 
said pressure data point associated with a respective 
one of said position data points; and 

(ii) mapping each of said pressure data points into a 
corresponding distance data point, said distance data 
point including a specified location for positioning 
an associated position data point; and 

said step (c) further comprising the step of: 

(i) utilizing said position data points and said distance 
data points to generate said set of parallel curved 
images. 

24. An apparatus for creating a drawing, comprising: 
one or more drawing tools for use by users of a client 

computer to draw a curve, each drawing tool having 
one or more style attributes associated therewith; 

an input device for receiving a plurality of location data 
points representing a path of a curve drawn with one of 
the drawing tools; 

a curve engine, coupled to the input device, for mapping 
the location data points into a plurality of location 
curve data points, the location curve data points used to 
display the curve in accordance with a specified image 
resolution; 

a display device for displaying said freehand sketched 
input in accordance with the style attributes; 

an image generation mechanism, coupled to the curve 
engine and display device, for mapping the location 
data points into an image of a curve in accordance with 
the drawing tool used to draw the curve; 

the curve engine generating one or more sets of location 
curve descriptors, the location curve descriptors repre- 
senting a compact representation of the location curve 
data points; and 

a communication device for transmitting the location 
curve descriptors from the client computer to a server 
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computer for retransmission to at least a second client 
computer, and for receiving location curve descriptors 
from the second client computer via the server com- 
puter; 

the image generation mechanism including a mechanism 5 
for converting the location curve descriptors received 
via communication device into a plurality of location 
curve data points, and using the location curve data 
points to display the curve in accordance with a speci- 
fied resolution and in accordance with a specified one 10 
of the drawing tools. 

25. An apparatus for creating a drawing, comprising: 

one or more drawing tools for use by users of a client 
computer to draw a curve, each drawing tool having 
one or more style attributes associated therewith; 35 

an input device for receiving a freehand sketched input, 
the freehand sketched input including a plurality of 
location data points representing a path of a curve 
drawn with one of the drawing tools; 1Q 

a curve engine, coupled to the input device, for mapping 
the location data points into a plurality of location 
curve data points, the location curve data points used to 
display the curve in accordance with a specified image 
resolution; 25 

a display device for displaying said freehand sketched 
input in accordance with the style attributes; 

an image generation mechanism, coupled to the curve 
engine and display device, for mapping the location 
data points into an image of a curve in accordance with 30 
the drawing tool used to draw the curve; 

the curve engine generating one or more sets of location 
curve descriptors, the location curve descriptors repre- 
senting a compact representation of the location curve 
data points; 35 

wherein 

the received freehand sketched input includes a plurality 
of temporal data points representing a speed at which 
portions of the curve was drawn; 40 

the curve engine maps the temporal data points into a 
plurality of temporal curve points, each of the temporal 
curve points associated with a corresponding first loca- 
tion curve data point; and 
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the image generating mechanism utilizes the temporal 
data points to display the curve in accordance with the 
drawing tool used to draw the curve. 

26. The apparatus of claim 25, wherein the temporal data 
points are calculated from a distance between consecutive 
location data points. 

27. An apparatus for creating a drawing, comprising: 
one or more drawing tools for use by users of a client 

computer to draw a curve, each drawing tool having 
one or more style attributes associated therewith; 
an input device for receiving a freehand sketched input, 
the freehand sketched input including a plurality of 
location data points representing a path of a curve 
drawn with one of the drawing tools; 
a curve engine, coupled to the input device, for mapping 
the location data points into a plurality of location 
curve data points, the location curve data points used to 
display the curve in accordance with a specified image 
resolution; 

a display device for displaying said freehand sketched 

input in accordance with the style attributes; 
an image generation mechanism, coupled to the curve 
engine and display device, for mapping the location 
data points into an image of a curve in accordance with 
the drawing tool used to draw the curve; 
the curve engine generating one or more sets of location 
curve descriptors, the location curve descriptors repre- 
senting a compact representation of the location curve 
data points; 
wherein 

the received freehand sketched input includes a plurality 
of pressure data points representing a pressure at which 
portions of the curve was drawn; 
the curve engine maps the pressure data points into a 
plurality of pressure curve points, each pressure curve 
point associated with a corresponding curve data point; 
and 

the image generating mechanism utilizes the pressure data 
points to display the curve in accordance with the 
drawing tool used to draw the curve, 

***** 
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